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REMOTE DISPLAY CONTROL OF VIDEO/GRAPHICS DATA 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

[0001] The present invention relates generally to data exchange and processing thereof; 
more particularly, the present invention relates to a method and apparatus where a source 
delivers graphics data to a target, which receives and displays the graphics data, particularly 
where the source of the graphics data controls how the graphics data is to be displayed at the 
target. 

2. Background of Related Art 

[0002] Nowadays, a plethora of products exist for receiving and/or displaying video signals, 
e.g., television broadcasting stations, satellite broadcasting services, video feed services, or the 
like. These video signal receiver products may range from products that receive and process 
video signals only in analog form, to products that receive and process video signals in digital 
form, and to products that are capable of receiving and processing video signals in analog and 
digital form. Some products are equipped with other integrated features, e.g., recording 
capabilities, or the like. The available features for displaying (e.g., picture-in-picture (PIP), 
picture-in-graphics (PIG), double screen, or the like) the video signal also vary widely among the 
products. 

[0003] There are equally diverse products or sources for delivering the video signals, e.g., 
over-the-air broadcasting stations, cable operators, direct satellite services, live video feed 
services, or the like. Many of these sources send, in addition to the video signals, graphics 
data to be displayed at a receiver for providing programming guide, advertisement, value added 
services (e.g., stock quotes, news service, etc.), or the like. Graphics data can be in any visual 
form, e.g., a text, a still picture, an interactive menu screen, or the graphics data may itself be 
another video stream. 

[0004] For example, Fig. 1 A shows a conventional video/graphics data transmission and 
reception arrangement 100, where a source of video signal, an AV source 101 sends graphics 
data together with the video signal to a receiver product, e.g., an AV target 102. Conventionally, 
the graphics data and video signal are combined by a mixer 104 in the AV target 102, and the 



combined signal is sent to a display 103, which provides a display of the graphics data and/or 
the video to a viewer. 

[0005] However, in the conventional video/graphics data transmission and reception system 
thus described, because the AN source 101 does not send information regarding how the video 
and graphics data are to be combined, the AN target 102 must decide how the graphics data is 
to be displayed, particularly in relation to the video signal, which may be simultaneously 
displayed with the graphics data. Thus, the mixer 1 04 combines the received video signal and 
the graphics data without knowing how the source of the graphics data, e.g., the AN source 
1 01 , intended to display the graphics data. The mixer 1 04 typically resorts to the most simple 
method of combining the two signals, e.g., simple overlaying, even if the AN target 102 is 
capable of more advanced display features, e.g., the PIP, PIG, Double Screen, or the like. 

[0006] Specifically, in the case of a digital television (DTV), which may be, e.g., connected 
to a set-top-box (STB) via an IEEE1394 serial interface (as specified by "IEEE 1394, Standard 
for High Performance Serial Bus", published 1995, which is incorporated herein by reference), 
the DTV receives the graphics data and the video signal (i.e., an MPEG transport stream) from 
an STB. The transmission of the video signal and the graphics data is governed by standards, 
e.g., in the United States, The Electronics Industries Alliance (El A) Standards, EIA-775-A, "DTV 
1394 Interface Specification," published April 2000, and/or EIA-799, "On Screen Display 
Specification," published June 1999, both of which are incorporated herein by reference. 
Because these standards do not provide means for the STB to send control information with 
respect to the manner in which the graphics combined with the video signal or by itself are to be 
displayed, the task of deciding how the two displays are to be proportioned and combined with 
respect to each other is nevertheless still left up to the receiver product, i.e., the DTV. 

[0007] The existing standards at best provide overlaying the graphics data display over the 
video display by allowing specifications of "pixel definition," which may define the location, size, 
and gain factors of pixels for the graphics data, and thereby allow simultaneous display and 
blending of the two displays, i.e., by attenuating the graphics data so as to make the graphics 
data appear somewhat transparent in the foreground with the video display visible in the 
background. The pixel definition, however, is part of the graphics data, and does not define the 
relationship between the video display and the graphics display. Moreover, these conventional 
systems heretofore do not allow the source of the graphics data to specify how the video display 
is to be proportioned in relation to the graphics data display. Accordingly, despite the fact that 
many conventional digital television sets are capable of advanced display features, e.g., the PIP, 



PIG, Double Screen, chroma keying, color keying, or the like, these features are not utilized to 
allow the DTV to know how the graphics data is intended to be displayed. Thus, the DTV in a 
conventional system typically performs a simple overlaying of one display, e.g., the graphics, 
over the other display, e.g., the video signal, and the graphics data/video signal is displayed as 
shown in Figs. 1Aand 1B. 

[0008] Thus, the receiver product may display 105 only the graphics data over the entire 
screen as shown in Fig. 1 B, or at best simultaneously display 106, 107 the graphics data and 
the video signal in a manner in which the graphics display covers, and thus, obscuring a portion 
of the video display as shown in Fig. 1C. 

[0009] The above-described conventional graphics/video transmission and display method 
and system do not utilize available display features of the receiver/display product, and thus, do 
not provide the most effective and the most user-friendly display of the graphics/video. 

[0010] Moreover, the conventional method and system heretofore does not allow the source 
of the graphics data, which can determine a most desirable way in which the graphics data is to 
be displayed to specify how the graphics data display is to be proportioned with respect to the 
video display to be presented to a viewer. 

[0011] Thus, there is a need for a system and method of transmitting and displaying 
graphics data that allow a user-friendly display of graphics data with the most efficient utilization 
of the display capabilities of the receiver/display product. 

[0012] There is also a need for a system and method of transmitting and displaying graphics 
data that allow the source of the graphics data to determine, at least in part, how the graphics 
data is to be proportioned in relation to the video display and ultimately displayed to a viewer. 

SUMMARY OF THE INVENTION 

[0013] Various objects and advantages of the invention will be set forth in part in the 
description that follows and, in part, will be obvious from the description, or may be learned by 
practice of the invention. 

[0014] In accordance with the principles of the present invention, an apparatus transmitting a 
video signal and graphics data to a target device includes a controller generating a display 
control signal prescribing a manner in which said video signal is to be displayed in relation to 
said graphics data in a combined display and transmitting said display control signal to said 



target device, said combined display to be formed by combining said video signal and said 
graphics data at said target device based on said display control signal. 

[0015] In addition, in accordance with the principles of the present invention, an apparatus 
displaying a video signal and graphics data received from a source, includes: a mixer combining 
said video signal and said graphics data to form a combined display; and a processing unit 
receiving a display control signal from said source, to control said mixer by prescribing a 
manner in which said video signal is to be displayed in relation to said graphics data in said 
combined display. 

[0016] In accordance with yet another aspect of the principles of the present invention, a 
system displaying a video signal and graphics data, includes: a source of said video signal and 
said graphics data, said source transmitting a display control signal prescribing a manner in 
which said video signal is to be displayed in relation to said graphics data; and a target receiving 
said video signal, said graphics signal, and said display control signal, and combining said video 
signal and said graphics data in said manner prescribed by said display control signal to form a 
combined display at said target. 

[0017] In accordance with still yet another aspect of the principles of the present invention, a 
method of transmitting a video signal and graphics data from a source to a target, includes: 
sending a display control signal from said source to said target, said display control signal 
prescribing a manner in which said video signal is to be displayed in relation to said graphics 
data within a combined display formed by combining said video signal and said graphics data at 
said target. 

[0018] In accordance with yet another aspect of the principles of the present invention, a 
computer readable storage medium is provided having stored thereon a computer program for 
implementing a method of transmitting a video signal and graphics data from a source to a 
target, said computer program including a process of: sending a display control signal from said 
source to said target, said display control signal prescribing a manner in which said video signal 
is to be displayed in relation to said graphics data within a combined display formed by 
combining said video signal and said graphics data at said target. 

[0019] These together with other objects and advantages, which will be subsequently apparent, 
reside in the details of construction and operation as more fully hereinafter described and 
claimed, reference being had to the accompanying drawings forming a part hereof, wherein like 
numerals refer to like parts throughout. 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0020] Features and advantages of the present invention will become apparent to those 
skilled in the art from the following description with reference to the drawings, in which: 

Fig. 1 A shows an example of a conventional graphics data transmission/display system; 

Fig. 1B shows an example of a conventional display screen displaying graphics data in 
which the graphics data takes up the entire screen; 

Fig. 1C shows an example of a conventional display screen displaying the graphics data 
in which the graphics data display is superimposed over the video display, obscuring at least a 
portion of the video display; 

Fig. 2A shows a graphics data transmission/display system, in accordance with an 
exemplary embodiment of the present invention; 

Fig. 2B shows a display screen displaying the graphics data in which a picture-in- 
graphics (PIG) display capability of a receiver is utilized, in accordance with an exemplary 
embodiment of the present invention; 

Fig. 2C shows a display screen of the graphics data in which a double screen display 
capability of the receiver is utilized, in accordance with an exemplary embodiment of the present 
invention; 

Fig. 3 shows the graphics data transmission/display system shown in Fig. 2, in which an 
AN source and an AN target are connected to each other via an IEEE1394 serial bus, in 
accordance with an exemplary embodiment of the present invention; and 

Fig. 4 shows a flow chart depicting a graphics data transmission process, in accordance 
with an exemplary embodiment the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

[0021] For illustrative purposes, the principles of the present invention are described by 
referring to exemplary embodiments, particularly, with references to an example in which an 
implementation of the invention is shown in which the source of graphics data, for instance, a 
set-top-box and a receiver, for instance, a television set, are connected and communicate via an 
IEEE1394 serial interface. However, one of ordinary skill in the art would readily recognize that 
the same principles are equally applicable to, and can be implemented using any other types of 
graphics data sources and receivers communicating via other interface means, and that any 



such variation would be within such modifications that do not depart from the true spirit and 
scope of the present invention. 

[0022] In accordance with an exemplary embodiment of the present invention, a source of a 
video signal, which may be a broadcasting program, movies, or the like, may also send graphics 
data to be displayed together with the video signal to a target device. The graphics data 
represents information, which may be in any visual form, e.g., texts, still pictures, interactive 
menus, video streams, or the like, and may be related or unrelated to the video signal. The 
source of the video signal also sends control information relating to how the graphics data is to 
be displayed in relation to the display of the video signal. By way of an example, the video 
signal source may provide the relative sizes and proportions of the graphics data display and 
the video signal display in, e.g., a picture-in-graphics (PIG), a double screen display, or the like. 

[0023] In accordance with a particular aspect of the present invention, the source may query 
the target for the display capabilities of the target, and, accordingly, send appropriate control 
signals that efficiently utilize the available display feature(s) of the target. 

[0024] Fig. 2A shows the relevant portions of a graphics data transmission/display system in 
accordance with an exemplary embodiment of the present invention. As shown, the graphics 
data transmission/display system 200 includes an A/V source 201 , which provides the graphics 
data and the video signal to an AA/ target 202 via any communication unit known to those in the 
art. In addition, the AA/ source 201 may include a controller (not shown), which may be any 
microprocessor, micro-controller, or the like well known in the art, which generates a display 
control signal representing information regarding how the graphics data and the video signal are 
to be output to and displayed by the AA/ target 202. The display control signal may include, 
e.g., an identification of a display feature (e.g., PIG, PIP, double screen, or the like) to be utilized 
for displaying the graphics data and the video signal, respective relative sizes and/or 
proportions of the graphics data and video displays, or the like. 

[0025] In accordance with an exemplary embodiment of the present invention, the A/V 
source 201 may receive from the A/V target 202 information regarding display capabilities, e.g., 
a screen size, resolution, or the like, and/or available display features, e.g., PIG, PIP, double 
screen, color and/or chroma keying, or the like, of the A/V target 202. The information may be 
provided by the A/V target 202 upon a request from the A/V source 201 , or the information may 
be automatically sent by the A/V target 202, e.g., periodically, or upon a predetermined event, 
e.g., during a reset/power up, upon an initial connections between the A/V source 201 and the 
A/V target 202, or the like. The A/V source 201 processes the information received from the 



A/V target 202 to generate appropriate display control signals to the AA/ target 202 that best suit 
the capabilities and/or features of the A/V target 202 according to the graphics data and the 
video signal also provided to the AA/ target 202, thereby providing the most user-friendly and 
efficient display to a viewer. Specifically, the source 201 determines a composite format of the 
video signal and the graphics data and the size of a screen to be displayed using the 
information received from the target 202. For instance, if the size and the resolution of a display 
unit or a screen at the AA/ target 202 (DTV) are 60 inches and 1 920x1 200, respectively, and the 
double screen function is supported, the source 201 refers to this information to generate an 
appropriate composite screen of the video signal and the graphics data that best suits the 
screen size and resolution of the AA/ target 202. In this instance, the screen composition is the 
same as that of FIG. 2C, to be later described. 

[0026] In one embodiment of the present invention, the information received from the A/V 
target 202 is stored in a storage unit (not shown) in the A/V source 201, and thus, need not be 
queried each time a particular graphics data is sent to the A/V target 202. In accordance with 
another aspect of the present invention, the information may be updated from time to time, e.g., 
periodically or upon a prompt by the A/V target 202 when any of the capabilities and/or features 
of the A/V target 202 are changed. 

[0027] A mixer 204 receives the video signal, graphics data and the display control signal, 
and combines the video signal and the graphics data to form a composite signal according to 
the display control signal, and the mixer 204 sends the composite signal to a display unit or 
screen 203, which displays the composite video and graphics to a viewer/user of the A/V target 
202. 

[0028] Although the example shown in Fig. 2A shows three distinct signal paths for the video 
signal, graphics data, and the display control signal, it should be readily apparent that the 
signals may be transmitted using any transmission method, e.g., over a serial 
communication/transmission line. In one embodiment of the present invention, the video signal, 
graphics data, and the display control signal is sent over a serial interface, e.g., an IEEE1394 
serial interface, as packet(s) of bits as is familiar to those skilled in digital communications. In 
addition, a packet may include any combination of the video signal, graphics data and the 
display control signal, or in the alternative, each of the video signal, graphics data and the 
display control signal each include a separate and distinct packet. 

[0029] Figs. 2B and illustrate some of the advantages of the present invention. Fig 2B 
illustrates the display unit 203 displaying the graphics data produced by the A/V source 201 



enabling the picture-in-graphics (PIG) feature through the display control signals. In contrast to 
Fig. IB, graphics data 205 and video data 206 are simultaneously displayed, and at the same 
time, the size and/or position of the video display window is set according to the display control 
signals from the A/V source 201 In accordance with an aspect of the present invention, the 
video data 206 may be presented as a viewer selectable icon, which, when selected causes the 
video data 206 to appear on the screen, 

[0030] Fig. 2C illustrates a display unit 203 displaying the graphics data produced by the A/V 
source 201 enabling the double screen feature of the A/V target 202 through the display control 
signals from the A/V source 201 . In contrast to Fig. 1C, a display simultaneously displays the 
entire graphics data 207 and video data 208 in respective portions of the display unit 203 
without the graphics data 207 obscuring portions of the video data 208, thereby allowing the 
viewer to appreciate the entire content of the video data 208. 

[0031] As can be appreciated, the system in accordance with the principles of the present 
invention as thus far described, allows the A/V source 201 to process and define how the 
graphics data is to be displayed in relation to the video signal. As a result, the A/V source 201 
provides a viewer-friendly display and/or graphical user interface (GUI), which allow efficient use 
of the display capabilities and/or display features of the A/V target 202. 

[0032] Further features and advantages of the present invention will now be described 
referring now to Figs. 3 and 4, which illustrate an exemplary embodiment of the present 
invention in which the source of the video/graphics is a set-top-box (STB) and the A/V target is a 
digital television set (DTV). 

[0033] A system 300 shown in Fig. 3 includes an STB 301 and a DTV 302, communicating 
with each other via an IEEE1394 serial interface 309. Although in this example, the IEEE1394 
interface is described, it should be readily apparent to and understood by one of ordinary skill in 
the art that any communication interface, standard and/or protocol, may be used without 
departing from the true spirit and scope of the present invention. Communication between the 
STB 301 and the DTV 302 may be governed by The Electronics Industry Association (EIA) 
Standards, EIA-775A and/or EIA-799, by way of example and not as a limitation. 

[0034] The STB 301 receives from the DTV 302 via the IEEE1394 serial interface, feature 
information that provides information regarding display capabilities and features of the DTV 302. 
The STB 301 sends to the DTV 302 the graphics data, the video signal (e.g., an MPEG-2 



transport stream), and the display control signal, which is based on the display features and/or 
the display capabilities of the DTV 302 received therefrom. 

[0035] The DTV 302 may further include a video decoder 308 for decoding the received 
video signal to produce a decoded video signal. By way of example, the video decoder 308 
may be an MPEG-2 decoder. The graphics data is received and processed by a graphics 
processor 306, which produces, from the received graphics data, processed graphics data that 
is compatible in format with the decoded video signal output from the video decoder 308, 
allowing the processed graphics data to be mixed together with the decoded video signal. 

[0036] The DTV 302 may further include a mixer 305 that receives the decoded video signal 
from the video decoder 308 and the graphics data from the graphics processor 306, and 
combines the two signals to produce a composite video signal. The mixer 305 outputs the 
composite video signal to a format conversion unit 304, which converts the composite video 
signal, e.g., from analog to digital, and sends the converted video signal, e.g., RGB, CVBS, Y, C 
signals, or the like, to a display 303, which displays the video and/or graphics to a viewer. 

[0037] In addition, the graphics processor 306, the video decoder 308, and the mixer 305 
each are controlled by a CPU 307 in the DTV 302, at least in part, based on the display control 
signal received from the STB 301 . The CPU 307 also sends the feature and/or capabilities 
information of the DTV 302 to the STB 301 , e.g., upon a request therefore from the STB 301 . 
The CPU 307 or controller includes a processing unit such as a microprocessor or a like 
integrated circuit 

[0038] An exemplary embodiment of the process in which the system 300 of Fig. 3 may 
operate will now be further described with reference to Fig. 4. At operation 401 , the process 
begins, e.g., upon initial installation of the STB 301 or alternatively, upon power-up of the STB 
301 where memories are cleared, initial flag conditions are set, etc., as is well known in the art. 
At operation 402, the STB 301 requests and reads the display capabilities and/or the display 
features information from the DTV 302. In this exemplary embodiment, the request of the 
display capabilities and/or the display features information may include a <Open Descriptor/ 
Read Descriptor command available from an AudioA/isual Control Transaction Set (AV/C CTS) 
as defined in The AV/C Digital Interface Command Set General Specification, Version 3.0, 
published April 15, 1998, which is incorporated herein by reference. In response to the AV/C 
CTS command, the DTV 302 sends the display capabilities and/or the display features 
information to the STB 301 . 



[0039] The display capabilities and/or the display features information may include an 
information block with respect to the display capabilities and features of the DTV 302, and may 
take the form, e.g., as shown in Table 1 below. 

Table 1. Information Block 

CompoundJ_ength 
lnfo_Block_Type(0x0000) 
Primary_Fields__Length 
SpecifierJd(OOOOfO) 
PIG 

Video min X position 
Video max X position 
Video min Y position 
Video max Y position 
Video_Size 
Double Screen 
Left or Right Enable 
Chroma Keying 
Color Keying 



[0040] As shown in Table 1, the information block may include a <Compound_Length> field, 
an <lnfo_Block_Type> field, and a <Primary_Fields_Length> field, which collectively identifies 
size and type of the particular information block. A <Specifier_ld> field includes the identity of 
the supplier of the information, e.g., the manufacturer of the DTV 302. A field <PIG> indicates 
whether the DTV 302 supports PIG feature, or if the feature is supported by a PIG standard the 
DTV 302 follows. A <Video min X positions a <Video max X positions a <Video min Y 
position>, and a <Video max Y positions together specify a range of a definable position of the 
screen. A <Video Size> field indicates scale factors, e.g., 1/2, 1/8, 1/16, and/or the like, 
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supported by the DTV 302. A <Double Screen> field indicates whether the DTV 302 supports 
a double screen (or split screen) feature. A <Left or Right Enable> field indicates which side of 
the double screen the video signal is to be displayed. A <Chroma Keying> field and a <Color 
Keying> field specify whether chroma keying and color keying features, respectively, are 
supported by the DTV 302, and an appropriate syntax by which applicable color information 
may be described. It should be readily apparent that the width of each field in the information 
block may be different from the other fields, and may comprise any number of bits. 

[0041] Referring to Figs. 3 and 4, at operation 403, the STB 301 obtains the display 
capabilities and features of the DTV 302 from the information block received and react, and 
stores the capabilities/features information in a storage area, e.g., a memory, (not shown) STB 
301 . At operation 404, the STB 301 determines whether there is graphics data to be sent to 
the DTV 302. If there is no data to be sent, the process returns to operation 404 until such a 
time when there is graphics data to be sent. 

[0042] If at operation 404 it is determined that graphics data is to be sent to the DTV 302, at 
operation 405, the STB 301 reads the capabilities/features information previously stored in 
operation 403. At operation 406, STB 301 generates a display control signal that is appropriate 
based on the nature of the graphics data to be sent and the available display 
capabilities/features of the DTV 302. 

[0043] In this exemplary embodiment, the display control signal includes a control command 
following the syntax of a "vendor dependent command" as provided in the AV/C CTS. It should, 
however, be understood and readily apparent to those skilled in the art that any other signal 
providing display control information may be used in place of the above-described command(s). 
An exemplary embodiment of the control command is shown in Table 2. 

Table 2. Display Control Command 



AV/C CTS(OxO) 



Ctype 



Subunit 



Subunit Id 



Opcode(OxOO) 



CompanyJd(OxOO) 



CompanyJd(OxOO) 



CompanyJd(OxfO) 



PIG(0x00) 



Video X position(0x03) 



Video X position(0xE8) 
Video Y position(0x02) 
Video Y position(0x58) 
Video Size (00) 



[0044] The control command illustrated in Table 2 is a command that causes the DTV 302 to 
utilize the PIG feature, to place the video window at the position (1000, 600), and make the size 
of the video window to be 1/8. More particularly, an <AV/C CTS (0,x0)> field, a <Ctype> field, a 
<Subunit> field, a <Subunit_ld> field, and an <Opcode(0x00)> field together specify the type, 
e.g., a vendor dependent type, of the command. A <Company_ld> field identifies the vendor(s) 
that support the particular command. Two Video X position fields identify the most significant 
bits and the least significant bits, respectively, of a starting X coordinate of the video window. 
Two Video Y position fields identify the most significant bits and the least significant bits, 
respectively, of a starting Y coordinate of the video window. <Video Size> filed specifies a 
scale factor, shown to be 1/8 in this example, for a video window. 

[0045] It should be readily apparent to and understood by one of ordinary skill that other 
commands, e.g., double screen, chroma keying, color keying features or the like, may be 
provided using the vendor dependent command with the PIG command illustrated above or 
using any other suitable available commands. 

[0046] At operation 407, the formulated display control signal, the graphics data, and the 
video signal are sent to the DTV 302, which displays the video and graphics according to the 
display control signal received. 

[0047] Although in this particular embodiment the capabilities/feature information is stored in 
the set-top-box, it should be readily apparent and understood by those skilled in the art that the 
storage of the information may not be necessary if the information is obtained, e.g., periodically 
or even each time graphics data is sent. 

[0048] While the invention has been described with reference to the exemplary 
embodiments thereof, those skilled in the art will be able to make various modifications to the 
described embodiments of the invention without departing from the true spirit and scope of the 
invention. 
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